The VIVA Method: A Life-Cycle Independent Approach to KBS Validation

This paper describes the VIVA method; a lifecycle independent method for the validation of Knowledge-Based Systems (KBS). The method based upon the VIVA validation framework, a set of products by which a KBS development can be described. By assessing properties of these products, and properties of the links between the products, a framework for validation throughout the KBS life-cycle is defined. Introduction The VIVA1 Technical Annex (VIVA 92) identifies the needs for a Knowledge-Based System validation method which covers the whole of the development lifecycle. These needs arise from identified problems with software-based validation, which can be summarised as follows: ̄ It is not possible to determine the validity of a system from the software alone. ̄ It is not possible to ensure that an implemented system will be valid unless validation is carried out throughout the life-cycle. A validation method, designed to be integratable with existing KBS development approaches and which addresses these and other issues, is being developed in the VIVA project. This paper presents a brief overview of the method as it currently stands, and describes future work plans. Lloyd’s Register will build on the VIVA method to allow the assessment of both KBS, and systems containing KBS components. This will significantly increase Lloyd’s Register’s scope for software assessment in the future. I The Research reported here is being carried out in the course of the VIVA Project. The project is partially funded by the ESPRIT Programme of the Commission of European Communities as project 6125. The partners in this project are University of Aberdeen (UK), Computer Resources International (DK), CISI Inginerie (FR), European Space Agency (NL), Lloyd’s Register (UK), Logica and the University of Savoie (FR) The Derivation of Requirements for a Validation Method In order to determine requirements for a validation method covering the whole of the KBS life-cycle, an analysis of current KBS development methods was carried out early in the VIVA project. The development methods studied included VITAL (Kontio and Rouge 91), CommonKADS (Aamodt et al 92, de Hoog et al 92, Taylor et a192), KOD (Vogel 90) and the European Space Agency (ESA) KBS development guidelines (Allard 92). A number of differences were found, which are presented below. Ordering. There is no unique ordering of development phases between life-cycles. It was not always the case that lifecycle phase X came before life-cycle phase Y. For instance, the Knowledge Model development phase is defined in VITAL to occur after the Requirements phase, but in CommonKADS, it may come before or after. Transformation. There is no strict transformation between products ill development phases. The Knowledge Model, for instance, would not necessarily be transformed en masse into another product in any life-cycle. The inferencing may be transformed into part of the Functional Specification, but the Domain Knowledge may be directly coded. This would be the case if the domain knowledge consisted of an is_a hierarchy, and the implementation environment provided support for direct coding of hierarchies, as in, for instance, KEE. NonStandardisation. The above diffferences are compounded by the lack of standardisation in KBS life-cycles. The "same" development methodology differs between organisations, and even between projects. There are a great many "KADS" approaches, for instance. In order to define a suitable framework for the VIVA method, it thus became necessary to take a productbased view of the KBS life-cycle. By defining a minimum set of products which could be used to describe 102 l From: AAAI Technical Report WS-93-05. Compilation copyright © 1993, AAAI (www.aaai.org). All rights reserved. existing and future, KBS development approaches, the applicability of the VIVA method to any arbitrary development method should be ensured. The VIVA Validation Framework The VIVA method consists of two parts, shown below. ̄ A set of products by which to describe a KBS application. ̄ A set of steps describing how the products can be used to validate a KBS application. This paper concentrates on aspects of the products known as the VIVA Validation Framework (Wells 92). The products are describes in Table I. This framework defines the contents and features of the products and their constituent sub-products. The products in the set possess two types of properties, which form the basis of the VIVA method: ̄ Properties of Products. ̄ Properties of Interand Intra-Product links. KBS will be validated by assessing such properties using methods and tools developed on the VIVA project. Products in the KBS Life-Cycle Situation. A description of the environment into which the KBS will fit. An example would be an organisation model. Requirements. Functional and non-functional requirements of the system. Knowledge Model. A Knowledge Level model. Usually a description of an expert’s, or system’s, reasoning capabilities. b’hmctional Specification. A specification of the functional capabilities of a system. Design. A specification of the architecture, interfaces, data structures and algorithms to be used in the code. Code. The implemented software, including documentation. Table h The VIVA Validation Framework As the VIVA method will be applicable to any development life-cycle, it is not possible to fully describe all possible properties and techniques. What the method will provide will be a framework which can be adapted to the specific attributes of a life-cycle or development. This framework will allow the user of the method to describe the products, sub-products and links of an application to a level of detail which will allow the complete description to be user-completed. Figure I illustrates the level of support the method will provide. The products and sub-products in the Validation Framework are related to each other in one of two ways, structural or transformational. Structural links are usually, but not solely, internal to a particular product. Examples could be a crossreference in a requirements document, or a calling relation between functions. Transformational links are those links where a "source" product at one end of a link is used to generate the "target" product at the other. These are usually, but not exclusively, external, or between product links. Examples are the transformation of inference function designs into their implementation in code, or the transformational of a particular individual requirement into its corresponding functional specification. In practice, there is not a clear-cut distinction between structural and transformational links. Once transformed, a link often becomes structural. An example is the link between a design and its implementation in code. During maintenance, analysis is carried out at the design level, and then mapped on to the code. Thus, the design-code link has become structurah